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.ABSTBACT \ ^ s^.^ . ^ 

When, iipleienting an, inAovation, a multitude of 

tjoittonents' interact to change not only the users, but the innovation 
as well* This ^uide exple^ins the concept of innovation 
configurations, or adaptations' ia'de i*n innovations during 
iipir<«ntation. After presenting and' discussing past research on 
innefation c^ianges,. the report outlines a five step procedure for 
identifying these changes. A one year study testing thes<. steps is 
described* The repOft coticludes with a discussion of- probleirs arising 
ii itpletentetioh an^ evaluation cf an innovation, including 
evaluatot subiectivlty, degree of phange in tbe individual ^r • 
innovation, and the cosplete alteration of the innovation. (CJ» 
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Change is a major characteristio of our iiaodem world and the replaceitent 
of qld ii«ys of doing things wittv new methods is a cottnon sight In all institu- 
tions and organisations which wish to keep pace with society. Schools and 
colleges are n'b exception to this and most have sampled innovation products 
and ttrocesses in attempts. to solvevthe problem of how to best educate their 
students.* Vtifortunately* it has often been discov«!red that taany sophisticated 
ancy well^researched educational "iq^atlons" do not significantly io^rove 
student performance. V 
/ ifhen the outcome of a change effort Is measured and the conclusion Is that 
^there Is "no significant difference" In peifformance, the innovation naturally 
'takes the brunt of the blame. However, research on the impiementatlon of Ihnova* 
tions at the Texas Resealrch and Development Center suggests that the innovation 
may not always be at xault^ It is clear that educatlona^ innovations can become 
changed significantly during. implementation and their operational forms often 
bear little resemblance to the theoretical models for their developers. 



The research described herein was conducted under contract with the , 
National Institute of Education. The opinions expressed are those of the 
authors and do not necessarily reflect the position or policy of the National 
Institute of Education. No endorsement by the National Institute of Education 
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Jtn this paper the concept of limovat ion conf Igura t Ions is Introduced to - 
describe the adaptations made in innovations during implementation. A procedure 
for Identifying configurations is suggested &nd research. and evaluation studies 
utilising this procedure are referred to. The paper concludes with discussion " ' 

of issues that surface with understanding of the concept and implications for \ 

. . • ' ^ \^ 

.evaluatorst developers and facilitators. 



Background 



Over the past several years » the innovation has been the subject of much 
research in the field of education. Researchers first made an effort to define 
the term as it is used in education and to describe the characteristics of an 
innovation. Later, implementation of the innovation became a focus of study* 
and it was noted that innovations can become changed during the process. Most 
recently innovation research has centered around learning how to measure and 
evaluate the effects of implementation^ 



Innovation characteristics 

Ixady What is an innovation? Many researchers have approached thl« 

question. In their important review , of knowledga-utilisation studies, Roger* ^ ^ 

and Shoemaker (1971) offered the following definition of the tern^ iuiiovation: 

An ifinovatlow is an idea, practice, or object perceived as 
m%hy an individual. It matters little, so far as human behavior 
is concerned, wliether or not an idea is "objectively" now as 
measured by the lapse of time since Its first use or .discovery. 
It is the perceived or subjective newness of the idea for the 
individual that determine his reaction to it. If the idea seems ., 
new to the Individual, it is an innovation (p. 19). 

In an effort to describe innovations, Rogers anf Shoemaker proposed a set 
of pereeivod attributes which represent five distinct Innovation characteristics. 
These were extrapolated from the authors' review of past studies: (1) relative 



advantage, <2) compatibility, (3) complexity, (4) trlalability, and (5) observa- 
bility. * ' 

A note extensive list of lm:ovatlon diiaensions was proposed by Z^ltnan 
and tin (19^71). their twenty dinenslons relate the importance of change to the 
participants and the outcome of adoption. J^eir list includes such dimensions 
as cost (both financial and social), efficiency, perceived work, structural ; 

» » 

radicalness, divisibility of Innovation practice, and susceptibility to successive 
modifications. 

Contiiuiiikg the effort to describe the characteristics of iimaifatidns, Chiii 
and Downey (1973) developed ^a set of definitions relative to the type of chiMge 

* . 

that is produced. They identified four different approaches: (1) change in 

y. ' , 

terms of relationships to ^oals of the system, (2) change^ as any substantive 
difference in*the client system, (3) change by som^ theoretical model, and (4)" 
change In a methodological process of a system. 

the if facts of Implementation on the Innovation 

tl^ere is now increased' awareness' that change is a process, not an event. 
Only recently have researchers, policy makers, and practitioners all become more 
aware that implementation is a key phase in the change proceed (Berman, McLaughlin, 
Bass, Pauly & Zellman, 1977). Also, evaluators are discovering that what la 

% 

being implemented must be assessed directly (Hall & Loucks, 1977). If it is 
not, there is the danger of trying to measure a change that has not occurred. 
Charter and Jones (1973) referred to such sltutations as "nonevents". 

During the course of implementation, a multitude of variables interact 
to change not only the users, but the innovation ajs well. Different conceptual 
perspectives and terminologies hAve been proposed to describe an innovation as 
it undergoes change during implementation. Agarwala-Rogers, Rogers and Wills 
(1977) have proposed that the term te-invention be used to represent the "degree 



to uhlch an innovation is changed* in the process of implement at Ion (p. V)." 

i 

Ptobably the best known recent study that has Identified innovation 

adaptation is that published in several volumes by the Rand' Ck)rporation (Beraan 

t 

& Mcitaughlihv 1975), The authors report on their es^tensive analysis of policy 

♦ 

and system practice relative to the inp'lenentation of federal change agent projects. 
A key concept from the research is that of mutual adaptation, which is defined 
as "an organizational process in which an innovative plan is developed and 

,4/ 

modified in light of the realities of the institutional setting, and in which 

the organl^sati^n changes to meet the requirements of the innovative project" 

(Greenwood, Hann & McLaughlin» p. 31, 1975). 

Emrick, Peterson, and Agarwala-Rogers (1977) viewed the innovation as S; 

key variable when they evaluated the National Diffusion Network. As have others, 

they concluded that: * 

There is a trend for adopters (56%) to begin implementing 
only selected aspects of the innovation rather than the entire 
ittnovatlon. At the same tln^e, adopters will modify methods and 
materials in what they believe are reasoiiable ways, so as to 
make the innovation more consistent or compatible with local 
cpnditiona (p. 116-3) . ^ 

Burick* et al. point o^it that most of the modifications are viewed as small 

by thfe adopters, with less than 20Z reporting large changes, although in one 

site "the re'invention process was so complete that the innovation had been 

renamed and the staff insisted that they were not now adopting the developer's 

project, but had their own innovation (p. 119)." 

Sikorski, Turnbull, Thorn and Bell (1976) also recogniased the consequences 

of implementation for the innovation. Supporting the notion that change in 

the innovation is inevitable, they discuss how this can be dealt withs 

Recognising that adaptation may be an inevitable part of the 
change process, we believe that some change strategies should be 
geared directly to adaptive modification of innovations. This 
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oouX4 take plad« in tvro ways: (1) the developaent process could 
l4&«QXii4# AttentiOA to an Inaovatloo's susceptibility to adaptive 
«Sditte»ti0iii and (2) implenentatioti assistance could help users 
Mka fyilffiiitie adaptatiofia of inoovatloiis, 

Our^g dairalopneiit, the aaehanistt of field testing off era 
i* one oppdttmity to find out. how uaers might need to modify an 
. innovalion but to yield tHis information a field test would have 
to be fairly suatained»'*unlike the prevalent approach of giving 
uaera brief eiepetience with the innovation. 

Whitt uaeti are ^plementlng an innovation* the people assist- 
ing thMi sight work for iystematic adaptation gather than replication 
of i modal* ' In a rough way* each aite could Evaluate an innovation 
ittfatlvft1^«"beginning to Implement it, obsarving the result a, modify* 
ing althet the innovation or the implementaiion sttatagy, jbaarving 
thi new reei^ta and so on. Too much evaluation can atrangle an 
ina^ationp. of course^ but technlc^al assistance might offer aimple 
'fetSisda of gatharlng and analysing such information in order to make 
contimtotts inproveoents in the innovation (p. 19). 

fhls sttggeation to look closely at just how an ^bovatlon is being implemented 

tias hkmn pur sued by evaluators who are faced with aasesslng innovation effect a 

in real world settings* 



IvaXnatiftg toPly^ tat ion 

It can no longer simply be assumed that an innovation is in use in an 
unaltef ad form Just because it is supposed to be. In their evaluation atudiaa 
of follow Through Hodels» Stallinga and Kaskowltz (1974) were among the first 
to eatablish a set of procedures to verify, that an innovation waa actually in 
uae before collecting evaluation study data. In their study, innovation develop-* 
era were surveyed. for descriptions of what the innovation look like in 

use. "^he presence of these characteristics in study classrb9Bis was then documented. 

£vana and Scheffler (1974) also concentrated on the innovation and how it 
waa being implemented when they evaluated the Iddlvldually Prescribed Inatruction 
(IPI) mathematics program. In their work they attempted to assess which of 
eleven developer-identified innovation categories were in use at each atudy 
school. They discovered that not only did the degree of Ittplementatlon of the 
innovation (i.e., the number of categories observed) vary among schools, but 



that there were even differences in the categories. 

the probleos of how to measure the degree of implementation of an innovation 
are juet beginning to ha understood. Gephart (1976) has identified several 
classes of aeasureaent problems dnd more recently Owens and Haenn (1977) have 
developed a clear suommry of the difficulties. The latter authors also provide 
illustrations of how they have handled the problems In their 'evaluatlpn ,of 
Experience-Based Career:. Education (EBCB). 

In a study of what they call "program residuals", Mitroff and Boston (1977) 
looked at the use of innovations in a large urban school system after implementa- 
tion support under title I o^ ESBA had ended. They developed a procedure 
involving survey forms, questionnaires, and personal interviews to discover how 
and if innovations were being used a year after the end of implementation support. 

Fiillan and Pomfret (1977) have addressed myy of the issues related to 
measuring the innovation as a variable in the change process. They point out 
the complexity of understanding and measuring the changes in an innovation during 
isqilementation: / 

One of the most complex and /important issues regarding the 
formation of research instruments concerns the two perspectives 
of fidelity and adaptation. It may be that the fidelity per- 
spective with consequent specific idstrumwits, is most applicable 
i^hen itudyjUig the implemenUtion of prepackaged, relatively . 
explicit innovations. This approach is more questionable when 
innovations are at the early stages of development and use. Which- 
ever perspective one uses, it is advisable to view the measurement 
of the impleBentation as a "snapshot*' of ivhat users are actually 
doing with respect feo the innovation at one point in time. It is 
. important to consider that the nature of innovations in use may 
transform over time and that we need measKravents to detect these 
changes in further specification, redefinition, or development 
(p. 367). 

&arick» et al. (1977) came to a similar conclusion about the time factor in 
their NDK evaluation: 

The sample of adopters studied by observation or survey in 
this evaluation showed the implementation process to be very 
complex and uneven, trti^ch means that estimates of actual diffusion 
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Vaty considetably/ depending upon when assessments are made. 
Allio» actual implementation appears to be so gradual and cumulative 
tbat aAKfly asseisments of diffusion will probably underestimate the 
phenomifipn (p* 119-120)* 

fullan and P«^rat conclude: 

The main implication of this discussion is that researchers 
and praey^tionera in change situations should be oriented to 
addtaislie continuously the program explicltness and degree of 
COiiplexity of educational Innovations tha$ they are attempting 
. to. use (p. 371). >» 

PAEt liiaarch 

In an attempt to develop better understanding of the change process, 
the Procedures for Adopting Educational Innovations (PAEI) Program of the 
0nlv»rslty of Texas Research and Development Center for Teacher Education 
is tttudfing innovation implementation in schools and colleges. Although 
elsewhere eattensive study hiias been made of innovations from the point of view 
of dissemination and the adoptlon.decislon, much less is known about Innpvatiotti 
from the point of view of theJ- actual functioning at adopting sites. Thus, 
PAEI research has focused on innovation implementation at the user level* 

The conceptual basis for the PABI research Is the Concerns-Based Adoption 
ttodel (CBAH) (Hall, Wallace fi'Dossett, 1973). In the CBAM, the individual and 
the if2iiov»tion are the frame of reference Jrom which the change process is 
desctibed. The Individual and the innovation have been the cornerstones for 
PASI research as well. 

Two critical dimensions for describing innovation Implementation from the 
individual perspective have been Identified in the'CBAM and researched in the 
PAEI Program. These dimensions are Stages of Concern About the Innovation 
(Hall & Eutherford, 19*76) and Levels of Use of the Innovation (Hall, Loucks, 
Eutherford & Newlovc, 1975). Procedures for assessing the|| dimensions have 
been developed and a series of cross sectional and longitudinal studies have 
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been conducted, which have initially verified the existence of seven Stages of 
Concern and eight Levels of Use. 

The Stages of Concern dinension deals with how users (usually teachers) 
feel about an innovation from the time, they f i*st become awere of it until they 
have mastered it. It has been discovered that users are first concerned about 
how an innovation wUl affect them personally. Later concerns shift to how to 
complete the tasks related to using the innovation. Finally, users become 
concerned with the impact of the Innovation on pupils. 

The Levels of Use dimension operationally defines the various atates of 
user behavior, that is, exactly what a user is doing «rlth an innovation. The 
levels begin vrith complete nonuse of an innovatflon and move to orientation 

* 

.about it. Preparation Is the next step, followed by mechanical tile. Eventually, 
use of ^n innovation becomes stabilised and routine. At this point, users may 
begin to refine their use of the Innovation. Later they may Intagrmte their - 
reflnements^with their colleagues. Lastly users msy reach a point where they 
reevaluate the use of an innorvation and begin to seek major modifications in it. 

It %i4h while conducting Stages of Concern and Levels of Use studies involving 
more than 800 teachers and professors that it became glaringly apparent to PAEI 
researchers that a single innovation is seldom operatlonalised in the same way at 
different field sites and in different classrooms. Indeed, it wss discovered 
that no two individuals were being exactly the same form of the innovation being 
studied, nor did they agree on the same operational definitions. While some- 
times the adapted forms of the innovation were only slightly different from each 
other, at other times the innovation was almost unrecognisable. 

It is clear from PAEI studies and other innovation research that there is 
a. need lor conceptualising and measuring the changes made in an innovation during 
ii^l^entation. The concept of innovation configariition3''h&B been developed to 

10 
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neet this need and to provide a framework for the dlscudaion of the issues 
brought to light by previous research and evaluation efforts. 

Innovation Configurations > The Idea 



Tie developer* of an innovation usually has a "atodel" form or foras of 

the innovation in nind* Whether specified as such or not, th^s aedel tfill 

^ntaln certain key characteristics or components that are essential to the 
innovation. 

la 

As the innovation is dissemlnate4 and the developer's aodel is translated 
ii>Ho' practice' In different classrooms, it may be onrecognicable. At the very 
least, one ot more components may be "adjusted" to fit local needs. In 4 
study of team teaching at the Texas R&O Center, for instance, the size of the 
team, ^ha number of grade leA«, what was planned as a team, and team member 
instructional responsibilities were all components that varied across sites. 
A similar study of instructional modules, the components of objectives, packagiog, 
instructor role, and alternative enabling activities ivere all identified ai 
dlmenalons along which the innovation varied. At different sites, different 

f 

components may be adapted. In different ways. Thus, any one innovation can be 
said to have several different operational fortes or innovation oonfigutAtiona . 

Iimovittion configurations ate the operatloBal patterns of the innovation 
that rasukt £rom aelection and use of different innovation component variations* 
The components will be different for different innovations, but will generally , 
Include characteristics of the Innovation such as materials and role and styles 
of the user and clients. Bach tomponent can be varied or adapted* Row the 

this paper the term developer Is used thrq^jg^hout to ref ei; to the 
Individual or group that originally conceived an innovation* The specific 
\ * Job function of a developer is imnaterial in this instance* 

\ '\ 

. . \ 

• , \ 

erJc ' 11 
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component variations are selected, how they ^re organized, and the way they 
are used by the actors result in different operational forms of the innovation 

ft 

or different innovation configurations. 

The key to Identifying innovation configurations is to first determine the 
components and the component variations that describe the innovation in use. 
The degree of Specif icity and the complexity needed ils best determined by . 
considering the use to be made of the information. An innovation developer 
may emphasize ten components while a practitioner may consolidate these to ^-^ 
three or four. Furthfcr, the innovation developer may, and often does, tolerate 
less variation within each component than the practitioner. 

A Procedore for Determining Innovation C onfigurations 

Developing a procedure for determining configurations began almost ** 
serendlpltously as PAEI Program staff were solving a Level of Use Interyiewing 
problem. Levels of Use interviewers were having difficulty determining whether 
oubjtfcts. were in fact using the Innovation in question. The Interview Dubjects 
would say they i*ere, but their description of what they were doing would not 
fit with* the iTiterviewer's general impression of what the innovation was^^ ^ 

. • « , • i 

• I 

supposed to be. 

The first solution used by the program staff was to develop a set of 
inniwation characteristics or component* based on the developer's descriptions. 
The Levels of Use interviewers would then make the use/nondse decision based on 
whether the interviewee used a significant number of the components on the list. 
As the research evolved, however, it became increasingly obvious that there 
was more to determining whether the innovation was in use. There were variations 
in the components which had to be documented as well. It was found that an 
innovation* configuration pattern could be developed for each user by identifying 
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the component variations they were using. Several configuration studies 
have since been conducted by PAEI staf*" and colleagues and the procedure for 
analyzing conf igutations has become more elaborate. 

Freseirtly, the procedure for determining innovation configurations includes 
five steps. This procedure is not meant to be the sin^lqua non of innovation 
configuration measurement, rat'.-ir it is one way that /ftos been, found useful and 
that in many aspects parallels the work of Evans and Scheffler xl97A) and 

* I * 

Owens and Haenn (1977). 

The five-step process for determining configurations is outlined in the" 
flowchart in Figure 1. The steps are: 

1. Xnterviwing developers and facilitators for essential components. 
The first step in the innovation configuration identification process is to 

determine what the innovation is "supposed" to look like in practice. When 

- • ' ^' 

there is a developer apart from the change facilitator (s) , the developer must 
be queried for the essential components pf the innovation. Often actual 
interviewing is impractical or impossible since the innovation could have been 
developed at a distant school district or research and development ^ncy, it 
could be the product of a publishing comi>any, or it could be the ideas of a 
particular individual that are ou|:llned in a book. Thus, it may be necessary 
to review whatever descriptive ma'terial i« available in books, pamphlets, and 
teachers' manuals to determine the "developer's" components of the innovation. 

Those responsible for facilitating use of the innovation on-site (e.g.., 
curriculum coordinators, deans, principals) will have their own ideas about 
essential components of the innovation and what will be emphasized during 
Implementation. Previous experience intervi^swlng facilitators (and developers, 
when available) has indicated three questions to' be most useful: 

(a) What would you observe when the Innovation is operational? 



(b) What would pebple be doing? \ 

• ■• ■ '1 

(e) What are the critical ccmponents of the innovation? 
Previous experience also indicates that it is often difficult for developers 
and facilitators to answer these queitions. This may be due to a tendency to 
think in terms of "what" the innovation is, rather than "how" it should be used. 

this first step results in a preliminary set of key comppnen'ts of the 

Innovatioii with sone suggested variations. For example, fa'cilitators might . 

require teachers of an individualized program to use specific prdgram materials 9 

t ' ■ . 

and tegroup children once a week. Tnus, two components of this program are 

materials and grouping. Use of different materials than those specified would 

be a^ variation of the materials component . The dally regrouping of children 

-would be a variation of the grouping component. 

2. Interviewing and observing a small, sample of users for variations. 
For this step a small (e.g., N-10 to 20) sample of users is selected that is 
estimated to represent a wide variety in use of the innovation* Facilitators , 
and building administrators may be good judges of where variations exist. The, 
interviews conducted and observations made should deliberately be very broad 
and open>ended in order to find extreme variations: An Interviewer might iisk , 
for a general description of how the innovation is used. If the components 
named by the developers/facilitators are not described, the interviewer ^should 
probe the user for information about each component. Observations should also 
be broad-based. An ethnographic methodology is suggest ej: every innovation- 
relate'^ action taken and conment made by the user is v^ltten down, as is a 
description of the physical elements present in the setting that relate to the 
innovation. These notes can later be formalized into a written report. 

To follow the same example of an individualized program, teachers may have 
de8cr:^bed their use of the innovation by talking about the tests and record- 

.14 
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keeping systems' used. These become additional components. 3ome teachers may 
use teacher^made tests while others may use program-developed tests. .These 
are two ppsslble variations under tl^e component of testing. Observations also • 

contribute to the list of coiq)onents and variations. For examplet teachers 

^ • 

could be observed using a textbook, which becomes another variation under the 
facilitator's nominated component of materials. As a result <ff this step, more 
components may be added to the list and more variations can.be identified under 
each component. 

3. Developing interview questions and interviewing. As a result of steps 
1 and 2, a tentative list of components and component variations Is developed'. 
In step 3*interview questions are0eveloped to probe users about each of. these 
components. Using the same example, some interview, questions for the individual-* 
ized program might be: 

(1) What materials do you use? 

(Z) How often do you regroup children? 

(3) «What tests do you use? 

(4) How do you keep records? 

Since the possibility exists that ^here.are more components of the innovation 
than have come to light in steps 1 and 2, the specific probes should be 
followed Up by an open-ended question such as, "Are there other things about- 
your use of the innovation tijf^ I haven' t- asked you about?" 

Interviews are then conducted with a large sample or all of the persons 

t 

involved in the study. The interviews should be taped or notes should be 
taken about use of each of the components of the innovation. 

4. Constructing a component checklist. Resulting from the preceding steps 
are a list of innovation components and a set of variations within each component. 
These can be formulated into a checklist. Using the same example, a part of a 
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checklist for the individualized 'program might be: 
Materials 

Uses progra^i materials 
. Uses a textbook 

Uses teacher-made materials 

Uses a combination of materials 

Record-Keeping 

Keeps records orf program card 

Keeps records on teadlllr-made chart 

* Keeps records in record book 



' .This checklist may be completed by the Interviewers for each user of an 

, innovation or" they may be filled out directly by the users. When the checklists 
' are completed, the different operational forms of the innovation being studied _ 
will be evident. 

5. locating dominant innovation configuration patterns . One useful 
p aspect of the configuration concept is that the various operational forms of a. 

j^given Innovation can usually be reduced to a manageable set of dominant innova- 
tion configurations. To do this, each variation for each component on the 

* * 

checklist is given a numbef; e.g. t 

* 

Materials 

9 

1 Uses program materials 
^ 2 Uses a textbook 

3 Uses teacher-made materials 

4 Uses combination of materials 

Record-Keeping 

1 Keeps records on program card 
' ^ 2 Keeps records on teacher-made chart 



\ . is 

. 3 Keeps records in record book 

O • 

Bach user is then assigned a code number containing as many di$it placet 
as they are conpotients in the innovation^ Bach digit place correspotids to 
a specific coppenent. The number in that place refers to the variation of ^' 
tha coi^ponent that waa used. For the two-component checldist above, (he user 
irith code 2-3 uses a textbook and keeps reeords in a record boolc. Each code 
rapreaafits an operational innovation configuration. Users with like codes 
can be grouped together. * 

Obviously, the more components an innovation has and the more vitiations 
there are within the components, the more possible innovation configurations 
there itill be. If an innovation is complex, it may be necessary to further 
condanae groups of users. To do this, judgements must be made about the 
importance of each component variation. Some variations can be collapsad while 
others may need to be kept discrete, depending on the ultimate use of the data. 
For exaiiple, if, "grouping" were a component, "small flexible groups" and "smII 
Stable groups," two variations, might be combine^ into "small groups." Sona 
components might be eliminated. Another exapple, in the checklist above, how 
a. teacher keeps records migl^t not be important, but that records are kept at 
all might be very important. In this reduction process some detail is lost, but 
the gain is a manageable number of innovation configurations which will account 
for a large number of classrooms. The next section describes a research study . 
where such an innovation configuration checklist was developed and data reduction 
wai accomplished Msing this procedure. 

Configurations j)f a Math Curriculum 

A full yaar study designed to devalop a procedure for identifying configura- 
tions was conducted utilizing a criterion-referenced mathematics curriculum for 
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kiAd«¥gavten through grade eight. Nineteen schools and 168 teachers participated 
in the study. 

^rogratt devel0pers« %iho were also district facilitators, gave three-basic 
titqulresients for teachers of the prograsi teach the objectives, test the 
objectivfts, and keep records according to the objectives. Vflien teachdrs talked 
about the prograa they typically deseribed the materials they used and their 
irouping pattarns. Figure 2 shows the checklist that resulted from steps 1-4 
ol feha aottfiguration identification procedure. Figure 3 describes tha eight 
configurations that resulted from the checklist analysis in step 5; Subsequant 
analysis l^d to a revised innovation configuration checklist which can be 
coipleted by teachers (see Figure ^). This checklist has proven useful for 
district and staff development purposes. 

Several issues surfaced while this study was underway. Since it was 
eiepldratory and conducted by outside researchars* the developers and users 
oont^ribotad equally to noainatini items for the checklist. If the devfeloperU 
p^fpactlve had bt«n Wh«ai«ad, user variation in materials and grouping would 
not ^ave been included, since only the objectives, tests and record-keeping 
cdmponeiiti were important to the developers. Checkliet construct ion is obviously 
fttbjectiva and is influenced by the intended purpose as well as the perspective 
of the person constructing it. The point during the implementation process 
at which the datji for steps h % is cQllected can also* have a bearing on- checklist 
construction. 

- The reduction of checklist data to determine dominant patterns can also 
be rather subjective. Juat how different component variations are cdmblned 
or eliminated to make the resulting data more manageable will depend greatly on 
irho li doing the work and for what purpose. To reduce the math program data, 
thrit components were emphasiaed which seemed, at least to the program staff, 
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to tfipreeent the essence of what teachers were doing differently across the 
district. Some variations of these components were combined Into single categorie 
For example, for the "materials" component, "textbook only" and "textbook empha- 
sised^' were considered as single variation. Very different combinations and 
reduction* may have resulted if teachers or district facilitators had been 
involved in the process. 

k similar procedure was used by Susan Heck at the Southwest Educational 
Development Uboratory (1977) to determine the operational configurations of a 
parent training package designed to teach low-iticome parents certain essential 
parenting skills. Reidy and Hbrd (1978) also determined operational conflgAira- 
tions for a mathematics skills achievement monitoring program in Fitchburg, 
Masaachuaetts. They also found differences in learning outcomes that could be 
associated with different innovation configurations. 

' Implicationa and Discussion 

Configuration Continuum • 

As has been seen, any one^ Innovation can be and usually is operationaliBed 
. in many differlkt forms. How different are these operational forms from the 
dev^opers' model («)? When has an innovation been adapted so much during 
implementation that it is no longer the same innovation? To answer these and , 
other questions, it may be useful to consider the various configurations of an 
innovation as lying on a continuum. One such continuum is illustrated in Figure 
5. In this figure the innovation is. a car. Configuration components ^oip.d '" 
include a number of doors, engine, interior styling, etc. At^he far right pf 
the continuum lies the 2>eVelop©r's Model, The djyel^per may restrict the model 
to one tightly defined operational fopaTe.g., four-door Fo^d), or the developer 
may allow for some variat^n Ce^g., four door GM products are o.k.), or perhaps 
even encourage usef^ adaptation of innovation components (e.g., Continental Mark 
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tV and VW bugs are appropriate also). However, most developers will reach a 
point beyond which, the adaptationw made to their innovation will not be acceptable* 
Beyond this point on the configuration continuum, the point of drastic mutatiorit 
the developer will not accept what is being used as the innovation. 

The point of drastic mutation naturally varies from innovation to^nnovation 
Since some developers rtquite a stricter adherence to their model than others do. 
Users and facilitators often have their own ideas about when an innovation has 
been adapted too mith and they may determine different points of drastic mutation. 
Figure 5 represents a case where the innovation developer is relatively conserva- 
tive In specifying what constitutes use of the innovation [see Innovation j 
Developer's Point (IDP)], while the change facilitators and staff developers 
[see Change FaciliUtor*s Point (CFP)] were more liberal in determining acceptable 

* 

jconfigurations, and users were still more liberal [see User's Point (UP)]. Other: 
relative positions could have been drawn. For example, users or^changc^cili- 
tatore may set more rigorous .standards than the develoj^s^tttothing but Honda 
Civics) . 

Thus* depending on the^oiei^roup, there may be a difference in opinion 
about whether a ce^tainconf iguration of the innovation does or does not 

anaccep table operational form. Rather than there being one "point" 
of drastic mutation, there is a zone or' area of drastic mutation. Within this 
•area there is disagreement about what constitutes use of the innovation. The 
more- conservative or liberal one role group is in setting their decision point 
relative to another, the larger area of disagreement. 

Another way of approaching this question is in terms xif fidelity of 
implementation. The followinjg definition of fidelity has been suggested by 
R. Wallace (personal correspondence, 1977): 



Purity of implementation - Number of components observ ed 

Number of components in developer's model 
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With this education, a user implementing dperational form of an Innova- 
tion identical to the developer's model (ct.g., tour-door Ford) would have a 
fidelity value of .one* Users' who have altered cr adapted so • many components 
that their forw of the innovation lies beyond tjte area of drastic mutation 
<e«g*; helicopter) would have a fidelity value of zero. 

I Unfortunately, everyone may not describe the innovation using the same 
components. In the continuoustprogress aatheMff^ curriculum studied, the 
developers emphasised the objectives and testing in describing the innovation 
While the teachers^etfcribed the innovation in terms of materials and record- 
kee^^g-^fstems. With most innovations, there is not sufficient epecificity 
about components to ebsolutely determine when the point of drastic mutation has 
been reached or when the fidelity value equals zero. This problem could be 
corir^ted if more care were taken to accurately describe innovation components 
and limits to conponent variations. 

Present implementation practice implicitly places a great deal of responsi- 
bility on the user for determining the configuration that is operationalized 
"behind the classroom door." In many cases, developers and change facilitators 
do not make clear the minimum operational essentials (critical components) of 
an innovation. Users freely adapt the innovation with little or no knowledge 
of the potcailal conse^iuences . Who should determine how much and which variations 
are allowable (i.e., valid)? It seems clear that this determination should be 
made or at least anticipated prior to the time of implementation. 

Describlntt Innovation Components , 

Change facilitators and evaluators would have an easier time in determining 
if certain innovation configurations constitute valid use of an innovation if 
innovation components were more accurately described. Perhaps innovations 
should be described in terms of "critical components", a core of components 

21 
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uhlch «t«t be present for the Innovation to be in uee, and "related components", 
peTlpti*feral components Ifthat are of secondary Importance. 

It is now oowicm to use altogether different components to describe 
different innovations. In theory, though after, configuration components hav9 
been identified for a large number of innovations, it may be' possible to develop 
a set of generic e<mfiguration components, ^ome thinking has, already been done 
in this direction. Leithnood (1978) has identified eight "dimensions" for 
describing school curriculum. These begin with the "platform" or a system of 
assuif tions used as a basis, to decide what to include in the curriculum. Other 
diiiensions include such things as objectives, teaching strategies, and learning 
experisnce. 

-«» 

(aia«i^» in the Individual 'vs« ChanRe in the Innovation 

fhe'*eoneept of innovation configurations describes the innovation as it 

» 

is changed dating implementation. But how much people-change and how much ' 
ionovation^ctiange is appropriate in a given situation? *^ ■ * 

Different cofld>inations of people-change and innovation-change are , 
represented in Figure 6. These representations are illustrative of how the 
innovation configuration continuum can be contrasted with ivhat is happening 
to innovation useri. In Case A, the innovation is implemented in a user system 
where the developer* s model was closely adhered to and the user made large 
changes. In Case B» ^here is some change to both the innovation and to the 
user, while in Case C, the user changes very little and the innovation is changed 
a great deal. Case D represents a case where the innovation is changed beyond 
the i^int of drastic imitation and the user does not change at all« 

Ther' are alternate interpretations of these cases but the point to be • 
emphasised is ttot change facilitators are responsible for determining explicitly 
or implicitly how much user change and innovation change will be accepted. 
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Depending on the goal of the change effort, a greatly modified forn of the 
innova^on laay be appropriate. In another case, the users may already be 
doing approximately what the developer's model requires so that the implementa- 
tion period will be short and the dteveloper*s model will be Implemented with 
little change. However, it should be noted that the Rand Implementation Study 
(Berman, McUughlin, Bass, Pauly & Zellman, 1977), concludes that small changes 
in the user are less likely to be long lasting. 

In an earlier Rand study report Berman and McLaughlin, (1975) fout^ that 
*'mutual adaptation" was the most successful change strategy. Ifcitual adiptation 
is represented In Figure 6 by Case B; the user changes some and the innovation 
is changed some. Although this may have been the most successful implementation 
strategy across federal diange agent projects, with some Innovations mutual 
adaptation could result in mutations that are -drastic enough to create less 
effective innovation configurations. If the critical components of an innovation 
must be closely adhered to in order to get validated learning outcomes, then 
perhaps mutual adaptation would not be the most appropriate strategy. 

Innovation Implementation and Evaluation 

Perhaps the innovation implementation process should vary depending on the 
completily of the innovation and hc^ close to the developer's model the imple- 
mented configurations should be. If closeness to the developer's model Is 
sought or the innovation is complex, more implementation facilitating activities 

♦ 

will be necessary as will a longer Implementation period. 

The more change an innovation represents from present practice, the longer 
the implementation period will probably have to be. And, If the deyelopers 
have not cl«^ified the critical components of an innovation, some implementation 
resources may be needed to clarify Just what innovation configuration is to be 
implemented. 
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Oftei^ it appears that developers will specify the inplementation requirements 

(e.g.» the! number of microscopes and time periods), but not include an operatiohal 

descriptloji of what will be obs.erved and what people will actually be doing when 

the Innovation is in use. This results in a greater potential for user confusion 

about what is expe$:ted and more deviation from the developer *s model. 

Two ' e]|.ementary science curricula. Elementary Science Study' (ESS) and 

Science A Process Approach (JAPA), ci^n be used to illustrate this point: 

y ESS appears to be an example of an innovation in science which 
has a variety of developer conf igurations and a great deal' of latitude 
for modification of the related components to meet site-specific needs. 
On the other hand, SAPA, because of its hierarehlal approach within 
and between grade levels, could be viewed as having, a less variable 
original conf£|Uration and a more restricted set of related components, 
(Roberts, pers'dnal correspondence, 1977). > 

So, an Implementation strategy for SAFA should emphasize the importance of 

adhering closely to the objectives and hierarchy while an ESS impleaentation 

strategy should emphasize diversity (e.g., user creativity) . in use of the 

* * 

innovation materials. ;^ 

Kb has been noted, innovation components are often not accurately describe% 
This raises sooev Important implications for implementation. It appears that 
when the developer Is not clear in describing an innovation or when change 
facilitators do not coMnunicate effectively, users are more apt to implement 
an unusual configuration. If, however, the implementation process is well 
designed and well articulated and is consistent with the developer's model, 
semantic and operational confusion is less likely to occur. 

Should complex Innovations which require the use of many critical components 
be implemented all at once or should they be phased into operation? The -t^fSUia 
of change will likely be reduced with_a.phaslHftg-iftfategy. There is, however, 
a chance that the-devgloper's^del may become lost in the gray area created 
the initial use of a less demanding configuration. 
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' Ii^anotber part of the PAKI/CBAM research, evaluators have been encoutaged 
to check for the uae or noause of an, innovation before meaauring outcomes (Hall 
k LottciTft, 1977). The concept of Innovation configurations suggests thAt 
evaXttatot^ may need to doctmeat carefully which critical and related components 
are btiilg used. The effects ohtalned from drastically mutated forms of an 
innovation aire likely to be different and may be less desirable than those 
^tained with the developer's model. Evaluators and developers could run ^ 
empS^ical teats on different configurations using planned varlfctions to determine 
ifhleh\components end combinatH^s of components dreate the most effective 

configurations for meeting the objectives of the IniAOvatlon. 

*\ • • , . 

Conclusion ^ 

Change is seldom easy. Most j)eople are reluctant to abandon trled-attd~ 
true methods for new ways if they are not certain that the new ways wli; be 
beneficial in the long run. Schools and colleges are no exception to this and 
many are now shying away from educational innovations because they have ditcovered 
that after the trauma of implementation is over there Is usually fto significant 
Improvement In student performance. It is clear now, however, that the innovation 
cannot always be blamed for this since it is often drastically altered during^, 
implementation. It is no longer safe to assume, as, moat people involved with 
educational Innovations aXways-ha^, that a pure form of an inaovation has been 

_lmpleme]&ted iust because the decision to do so has been made. 

If the impact of educational ihnovations arc to be evaluated fairly, it 

. must be dete mined just how they are adapted during implementation. The concept 
of innovation configurations deals with this problem and provides a simple but 
precise method for determining how the operational forms of an innovation differ 
from the model (s) of the developer. 



Figure 1 

A ptocedureT for identi£y:(.ng Innovation configurations. 
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^ Figure 2 

/ Original Itmov&t Ion configuration checklist for math curriculum study. 



k urn Of THE nVE DOTNSIONS below, check the one that msi CLOSELY DESCRIBES THE 
y-^miA* If NONE DESCRIBE THE TEACHER, USE THE SPACE ON PACE 2 UNDER THE APPROPRUTE 
^DlMl^StOII TO DESCRIBE tHE SITUATIOK IN . . 



t«.r l^itruetiotUil retoure«t usad 

. -- ProifMi a«t«rl«ls only 

" ■ . T«x£i>ook only 

^ T«ie)^book wiphMixed, Progr«« 

Mterlalt ■upplttaiental 

^ Prograai Mt«clals «npha8lzttd, 

~^ t<xt;book suppl^Mntal 

2, atouplng pAttcmt 

' Large gtoup within heterogeneous 

Glaesroom; whole date or two 
large groups 

•Large group within homogeneous 
classroom: whole elass or two 



Enriched conbinstion, including 
such sources as gaaes, sanip* 
ulative materials, learning 
centers, etc., ss well as the above 

Unknown « 

Other — describe on page 2. 



Snail stable groups 
Snail flexible groups 
Unknown 

Other — describe on page 2 



Isfge groups 

3« Progtan objectives used la.rgely in sequence ae guide fdlf Instruction 

yes No Unknown Other — describe ^n £ege 2 

4« Kind of. testing 

Posttrft on objectives ^(cither Teacher observation only 

progras-supplied or teacher-made) Unknown 

and cluster test (either program- ^ 9 

supplied or teacher-made) Other - describe on jjage 2 

Cluster test only (either program- 
supplied or teacher-made) 



5. Test utilization 



Test results detem^e what is to 
be taught next to each individual 

Ccoup goes on if most pass test; 
stragglers ere given speciel 
sttention 



Group goes on if most pees 
test; those who fall ere not 
attended to ■ 

Unknown 

Other — describe on page 2. 



Flg^uTje 3 ■ . 

• Innovation configurations for a nath curriculum. 

t 

t 

CoMwnalltlM t iai configuration pattcrnB have the following elements in common: 

Objectlvts: Program objectives are taught 

Testing: Students are evaluated on their 

achievement of , program objectives 
Record Keeping: .Teachers record achievement or non- 
achievement of program objectives 
Inaedlatelx after ev/iluat Ion occurs 

• « 

.Xk)nflguratlon 
Patterns: 

A (H ■ 11) Materials: Program materials emphasized 

Grouping: Large group (s) within homogeneous classrootf 

* ' Test Use: Group goes on if most paas test, attention to 

stragglers 

« 

B (N " 34) Materials: Enriched combination 

Groupings-. Large group (s) within homogeneous classroom 

Test Use: Group goes on If most pass test, attention to 
stragglers 

C <» - 16) Materials: Enriched combination 

Grouping: Small flexible, groups ^ ^ , 

Test Use: Test results determine next steps for individuai. 

D (M - 14) Matatlalii Enrlehad combination ^ 

Grouping: IndlvlduallMd * ^ 

Test Use: Test results determine next steps tr-^ individual 

E <N " 8) Materials: Program materials emphasized 

Grouping: Individualized , ^^.^^...1. 

Test Use: Test results determine next steps for individuals 

F (N * 6) Materials: Textbook emphasized 

Grouping: Large group(8) /within homogeneous classroom 

Test Use: Group goes on if most pass test, attention to 
stragglers 

G (N " S) Materials: Enriched combination 

Grouping: Large group(8) within homogeneous classroom 

Test Usa:' Group goes on If most pass test, stragglers not attended, to 

H (N ■ 5) Materliils: Program xnaterials emphasized 

Grouping: Large group(s) within heterogeneous classroom 
Test Use: Group goes on if most pass test, attention to 
stragglers 
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Figure 4 

Revised Innovation configuration checklist lEor math curriculum study. 
t.D. (Last 4 digits of SS#> ^ 

k^ase check o^e choice under each of the five categories below that is the soat descriptive 
df your math instruction. 

i, toetruetionaX resources used: 

Prograit katerials (i.e., packets, worksheets) only^ 

• Textbook(s) only , ^ i ■ 

Textbook(s) emphasized, program materials supplataental 

Program materials emphasised, textbook(s) supplemental ^ a 

Co«£i&ation of text(#) and/or program materiala with teacher-made materials and 



^ Ur«t variety of text(s), program materials, games, teacher-made materials,. 

manlpulatlveiB, centers, labs, etc. 

" ■ ' ■ ' 

2. Grouping patterns t 
. Teach whole class or two groiips in a class with children ranging in abilities 

* (i.e., heterogeneous) ^ ^ ^.^n. * ™^,.n», 

Teach whole class or two groups in a class with chil^dren of generally the 

same abilities (I.e., homogeneous) ' , ^ w 

Teach 3 qr more small groups that are fairly stable-the children in each 



ftroup seldom move to a different group ^- -^ . 

Teach 3 ormore sma]l groups that change continually-«the children fraquently 



move to a different group. 
Teach individuals otly, no grouping 



3. Clusters of objectives: 
Um program clusters largely in •equence at a frammrork for instruction 
Vm program cluiters largely out of sequence 
Qo not use program clusters 

4. Object ivcs^^^^^ objectives largely in sequence within the clusters 
Use objectives largely out of sequence ' ' 
Do not use p^bgram objectives 

5. Kind of testing: 

Use posttest on objectives (either program-supplied' or teacher-made) And cluster 
test (either 'prograta-supplied or teacher-made) . . , 

Use cluster test (either program-supplied or 

Use posttest on objectives (either program-supplied or teacher-made) only 

Use teacher judgement only • , 4..4-*«.ni! 
Varies widely using posttests, cluster tests and/or teacher judgement 

6. The use of test results: 

_ Each individual child is assigned work or activity depending on the results of 

J Whcn^JSst^ojT the group passes a test the group goes on; those who fall are 

Sr«:rt1f tSrrroipVs^s a test the group goes on; those who fail will have 
"^TtiTt culi^^^^ due to the -Pj^V -tore o^,^^^^^^^ 

What Is done with test resulis depends on the objectives being taught 

ERJCE8I«B cote SlQilf ICANT mTURIS OF YOUR MATH Vf^GMH THAT WE HAVE NOT INaUDEDT 



Configuration Continuum 
Using "Car'' as the Innovation 



Not the innovation 

A 






UP CFP 





The Innovation 

A 



0ev<<(loper*8 
ModBl(8) 





Points of Drastic Mutation 
UP — User's Point 

« 

Cl^P Change Facilitator's Point 

DP Developer's Point 
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, . FIGURE 6. 

Cmm IN tHE USER AND CHANGE IN THE INNOVATION 



draatlc ehang e 




CHANGE IN THE INNOVATION 



dr«itle change 
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